wo 2005/086065 



1 



PCT/IB2005/050783 



DESCRIPTION 

Interactive computer-aided diagnosis 

5 

The present invention relates to the field of computer-aided data 
processing (CADx). In particular, the present invention relates to a method of computer- 
aided extraction of quantitative information, to a data processing device and to a 
corresponding computer program for a data processing device. 
10 Despite intense research activities in the field of computer-aided 

diagnosis, the transfer of resulting algorithms to the target market, i.e. clinical 
applications, is often not successftil. Methods of computer vision, automated 
classification, or comparable algorithmic solutions are not regularly used and even less 

regularly trusted by physicians. 
1 5 The m ost important reasons for such distrust toward research results 

include: 

- field testing in limited environments, 

- data processing methods remain incomprehensible to non-technical users, and 

- dependency of the quality of an algorithm bom technical parameter settings. 

20 Despite growing awareness of insuflBcient validations, inappropriate 

approaches such as, for example, described by Globus A. et. al. "14 ways to say nothing 
with scientific visualization" IEEE Computer 27 (7) pp. 86-88, 1994 are used. The 
computer vision algorithms are mostly validated by subjectiye visual impression of the 
researchers themselves. A quantitative confidence standard for all applications of CADx 

25 and similar applications is required to increase a well-founded trust of physicians 
towards these applications. 

WO 01/74238 Ai discloses methods of localizing a deviant region in a 
30 turbid medium to be used in optical mammography where a breast of a female body is 
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examined by means of light. Said methods produce images in which any deviations, for 
example tumors, can be recognized. 



5 It is an object of the present invention to provide for an improved 

computer-aided extraction of quantitative information. 

According to the invention, a method of computer-aided extraction of 
quantitative information, a data processing device and a computer program for a data 
processing device for performing a computer-aided extraction of quantitative 
10 information according to the independent claims are provided. 

According to an exemplary embodiment of the present invention, a 
method of computer-aided diagnosis is prov^ided wherein acquired first diagnostic data 
is processed on the basis of a furst parametex set to determine a first diagnosis result. 
Then, a confidence interval is determined with respect to the diagnosis result. The first 
15 diagnosis result and ttie confidence interval is displayed, for example, to a user. The 

first parameter set is then adjusted on the basis of an input, for example, fi-om a user of 
the system. Then, the diagnostic data is reprocessed on the basis of the adjusted first 
parameter set to determine a second diagno sis result which is displayed. 

According to this exemplary^ embodiment of the present invention, a 
20 multi-step-methodology is suggested which, is apphcable to a plurality of image- and 
signal processing algorithms that are often the base of clinical data processing 
applications. According to an aspect of this exemplary embodiment of the present 
invention, a confidence interval with respect to the fnst diagnosis result is provided. In 
other words, a feedback is provided to the uiser of the method which gives the user a 
25 feeling with respect to the dependability of the diagnosis. For example, for quantitative 
measurements, a 95% interval ("in 19 of 20 cases, the true result wiU be in the 
following range") may be part of the confin-ed presentation. Geometric results, such as 
segmentations or registrations of, for example, organs of a patient may instead of a 
statistic visualization of the confidence intcTval be displayed with an explorable "grey 
30 area" representing the area where the segmented or registered form may be varying. 
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By an appropriate input, the user may alter the first paranaeter set. Then, 
a reprocessing of the diagnostic data may be performed on the basis of the adjusted first 
parameter set to determine the second diagnosis result. From this, the user of the systena 
comes to know the influence of the changed parameter on the diagnosis result, and 
5 therefore the stability and reUability of the computed result. 

Advantageously, due to the confidence feedback, a trust in the system or 
method may be improved. Furthermore, due to the variation of the parameters, the user 
may interactively explore the influence of certain parameters or certain parameter 
ranges on the diagnosis result to thereby establish a feeling with respect to the 
1 0 dependability of the diagnosis result. 

According to another exemplary embodiment of the present invention, at 
least one parameter of the first parameter set may be varied by a user by a 
corresponding input. This may allow for an interactive reprocessing of the diagnostic 
data. 

■f 

1 5 According to another exemplary embodiment of the present invention, a 

distrust selection option is provided, for example, to the user of the system upon 
selection of which the first diagnostic data and the corresponding first parameter set is 
forwarded to a service port such that, for example, an R&D team may use this 
infomiation for further adjusting the method. For example, the selection of the distmst 

20 selection option may result in a storage of the data of the actual case in an anonymised 
data base. A customer advantageously may coUect this data and pass it on to marketing 
and R&D. This may allow that a user feels more empowered to increase the quality of 
the method. Preferably, the range of parameters is extracted firom clinical data acquired 
at the site of application, which enables each embodiment of the invention to 

25 individually adapt to clinical setups. 

According to another exemplary embodiment of the present mvention, a 
tmst selection option is provided upon selection of which the first parameter set may be 
stored together with the first diagnostic data. This set of first parameter set and first 
diagnostic data which has been proved to be appropriate may be used as ground truth 

30 for other diagnosis or may, for example, also be used for further training of the method 
or system. 
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According to another exemplary embodiment of the present invention, 
the first diagnostic data may be compared to second diagnostic data. A similarity of the 
jBrst and second diagnostic data may be determined and when the similarity meets with 
that criteria, the first diagnostic data may be reprocessed on the basis of a second 

5 parameter set originally belonging to the second diagnostic data. 

According to another exemplary embodiment of the present invention, 
the methods allow for an explorative detemiination of a dependability of at least one of 
three first and second diagnosis results. 

According to another exemplary embodiment of the present invention, a 

1 0 data processing device is provided allowing for a visualization of a confidence interval 
relating to a diagnosis performed on the basis of a parameter sets. Furthermore, this data 
processing device may allow for a reprocessing of the diagnostic data on the basis of an 
adjusted parameter set. This reprocessing may have the following effects. 

As a first effect, by adjusting the first parameter set, a second diagnosis 

15 result may be achieved which is more accurate. As a second effect, by adjusting the 
parameters of the first parameter set, a user may explore the dependability of the 
diagnosis result from the parameters. By this, the confidence interval may be visualized. 

The present invention relates also to a computer program for a data 
processing device for performmg a computer-aided diagnosis.. The computer program 

20 according to the present inventiori is preferably loaded into a working memory of a data 
processor. The data processor is thus equipped to carry out the method of the present 
invention. The computer program may be stored on a computer readable medium, such 
as a CD-ROM. The computer program may also be presented over a network, such as 
the WorldWideWeb and can be downloaded into the working memory of a data 

25 processor from such a network. The computer program may be written in any suitable 
programming language, such as C-H-. 

It may be seen as the gist of an exemplary embodiment of the present 
invention that a confidence interval relating to the dependability of the diagnosis result 
on certain parameters is visualized. On the basis of an input, the diagnostic data may be 

30 reprocessed on the basis of an adjusted parameter set to determine a second diagnosis 
result which is displayed on the display. Furthermore, standardized feedback 
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mechanisms, such as trust or distrust selection options may be provided allowing users 
to express their individual trust with respect to the method and/or system and to 
furthermore improve presented results by allowing that this feedback may automatically 
be used for further improving the method or system. 
5 Exemplary technical fields, in which the present invention may be 

applied, include baggage inspection, medical applications, material testing, and material 
science. Even in the case of medical applications, it is not necessary that the various 
methods steps have to be carried out by a physician. 

These and other aspects of the present invention are apparent from and 
1 0 will be elucidated with reference to the embodiment described hereinafter and with 
respect to the following drawings: 



Fig. 1 shows a schematic representation of a data processmg device 
1 5 according to an exemplary embodiment of the present invention which is 

adapted to execute a method according to an exemplary embodiment of 
the present invention. 

Figs. 2a and b show a flowchart of an exemplary embodiment of a 
20 method of operating the data processing device of Fig. 1 according to an 

exemplary embodiment of the present invention. 

Fig. 3 shows an exemplary embodiment of a display presented to a user 
according to the present invention. 

25 

Fig. 1 depicts an exemplary embodiment of a data processing device 
according to the present invention for executing an exemplary embodiment of a method 
in accordance with the present invention. The data processing device depicted in Fig. 1 
comprises a central processing unit (CPU) or data processor 1 connected to a memory 2 
30 for storing the diagnostic data, the parameter sets or any intermediate results. The data 
processor 1 may be connected to a plurality of input/output-, network- or diagnosis 



wo 2005/086065 



6 



PCT/IB2005/050783 



10 



15 



devices, such as MRI devices or CT devices via one or more data transfer mechanisms. 
The image processor is furthermore connected to a display device 4 (e.g. a computer 
monitor) for displaying information or images computed or adapted by the data 
processor 1 . An operator or user may interact with the data processor 1 via a keyboard 5 
and/or other output devices which are not depicted in Fig. 1 . 

Figs. 2a and b show a flow-chart of an exemplary embodiinent of a 
method for performing a computer-aided diagnosis in accordance with the present 
invention. 

The following exemplary embodiment relates to the field of computer- 
aided viewing. However, it should be noted that the present invention may be applied to 
a variety of application fields, such as, for example, computer assisted navigation in 
surgery, intervention planning, outcome control or others. 



After the start in step SIO the method continues to step SI 1 where 
diagnostic image data is acquired. The diagnostic image data may, for example, be an 
image fi-om a microscope. Then, the method continues to step S12 where a 
determination is made with respect to whether there is a ground truth for this diagnostic 
20 image data or not. In case it is determined in S12 that there is ground truth for this 

diagnostic image data, the method continues to step S16 where a first parameter set is 
set corresponding to third parameters of the ground truth. Then, the method continues to 
stepSn. 

The adjusted parameters of the parameter sets may comprise a 
25 significance level for statistical tests, thresholds for binary decisions, weighting factors 
for the combination of different processing steps, and scales and coefficients of image 
filters. In a further embodiment, additionally properties of the input image data like 
intensity, contrast, and noise intensity can be changed so that the stability of computed 
results to these image properties can be displayed to the user as well. 
30 In case it is determined in step SI 2 that there is no groxmd truth, the 

method continues to step S13 where it is determined whether there is comparable data 
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Stored in a memoiy or not. In case it is detemuned that tliere is comparable data in an 
accessible memory, the method continues to step S14 wlnere the first parameter set is set 
in accordance to second parameters that were valid for tbe comparable data. Then, the 
method continues to step SI 7. 

5 In case it is detennined in step SI 3 that ttaere is no comparable data, the 

method continues to step S15 where the first parameter set is set in accordance with a 
preset parameter range. Then, the method continues to st:ep SI7. 

In step SI 7 a computer-aided diagnosis is performed on the basis of the 
first parameter range. Then, the method continues to step SIS where the diagnosis result 

10 is displayed to a user of the system, for example on display 4. Then, the method 
continues to step S19 where a dependability or trustwortliiness of the displayed 
diagnosis result may be explored by a user by adjusting variables of the first parameter 
set. The adjusted variables are used to reprocess or re-perform a diagnosis to determine 
new diagnosis result. The new or adjusted diagnosis resmlt is then displayed to the user, 

1 5 for example via display 4. 

The user may, for example, adjust variables by moving a slider by means 
of a pointing device, such as a mouse pointer on a displa.y. Also, for example, a track 
ball, mouse or mouse wheel may be used to adjust the variables. 

Advantageously, by adjusting the parame-ters, the user may explore the 

20 dependability or trust worthiness of the diagnosis result. In particular, the dependability 
of the diagnosis result may be explored with respect to ttie adjusted variables. In other 
words, by exploring the dependability or taiist worthiness, a quantitative confidence 
measurement may be conveyed to the user of the system. . 

In a variant of this exemplary embodiment of the present invention, is 

25 also possible to provide quantitative confidence. For example, the methods may include 
visual feedback on the result probability distribution du& to algorithmic uncertainties. 
For example, for quantitative measurements, a 95% interval ("in 19 of 20 cases, the true 
result will be in the following range") may be part of the: confined presentation to the 
user on display 4. Geometric uncertainties with respect to geometric results obtained, 

30 for example, by a segmentation or registration may advaaitageously be not statistically 
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visualized by using numbers but with an explorable '^grey area", such as, for example, 
depicted in Fig. 3. 

During the development and the initial configuration of the method, not 

< 

only a set of technical parameters is determined which delivers in the middle the best 

S result but for all available training data an individual parameter set is detemiined which 
reproduces a known gold truth in a most appropriate manner. From this, for each 
parameter, a variance range may be provided or displayed to the user. Then, as 
described above, the user may use an input means, such as a mouse, mouse wheel or. a 
windows slider displayed on the display to adjust a parameter within this variance 

10 range. Advantageously, according to this aspect of the present invention, the user may 
directly (interactively) follow the changes to the diagnosis result on the display. 

Then, as indicated by the encircled A at the bottom of Fig. 2a and at the 
top of Fig. 2b, the method continues to step S20 where it is determined whether there is 
a distrust entry or not. A user may input a distrust input indicating that in spite of all 

1 5 adjustments made in step S 1 9, no trust has been built up to the diagnosis result i.e. the 
user does not believe that the diagnosis result is appropriate. In case it is determmed in 
step S20 that there is a distrust entry, as indicated by the encircled B at the top of Fig. 
2b and at the bottom of Fig. 2a, the method contmues to step S2 1 where the first 
parameter set is adjusted. This may be done by, for example, selecting a different 

20 processing method for steps S17 to S19 or by loading another set of first parameters. 

According to a variant of this exemplary embodiment of the present 
invention it may advantageously be possible to automatically forward the diagnostic 
image data and the corresponding first parameter set which was distrusted by the user to 
a service port from which it may be downloaded to a R&D department. This data may 

25 then be used to further improve the first parameter set to achieve better results. 

In case it is determined in step S20 that there is no distrust entry, the 
method continues to step S22 where it is determined whether there is a trust entry or 
not. In case it is determmed in step S20 that the trust selection option was selected i.e. 
there was a trust entry, the method continues to step S23 where the diagnostic image 

30 data and the first parameter set is added to the comparable data and/or the ground truth 
and/or is used for adjusting the preset range in step S21. As may be taken from Fig. 2b 
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and as indicated by the encircled C at the top of Fig. 2b and at the bottom of Fig. 2a, the 
method continues from step S23 to step S21 or to step S13 or to step S 12. In a variant, 
the method may stop at the encircled C, i.e. after step S23. 

In case it is determined in step S22 that there was no tnast entry, the 

5 method continues to step S24 where it is determined whether a leamin.g selection option 
has been selected i.e. whether there is a learning entry or not. In case it; is determined in 
step S24 that there is a learning entry, the method continues to step S2 5 where training 
examples are used to adjust the first parameter set These training examples may be the 
diagnostic image data and the corresponding first parameter set added to the memory in 

1 0 step S23 of a previous application run where the result was trusted by -the user. Training 
is possible whenever there are means to determine the true result other" than by the data 
processing method, e.g. by an expert opinion or other diagnostic data acquisitions. As 
indicated by the encircled D at the top of Fig. 2b and at the bottom of rig. 2a, the 
method continues from step S25 to S15. In a variant, the method may also stop after 

15 stepS25. 

In case it is determined in step S24 that there is not learning entry, the 

method continues to step S26 where it ends. 

Advantageously, as may be taken from the above description, the user 

may improve the CADx system by him- or herself when there is a corresponding set of 
20 ground truth and image data. For example, this set of ground truth and image data may 

be acquired from an alternative parallel diagnosis. Thus, for example, "the user may 

acquire a ground truth with respect to certain diagnostic image data from an intervention 

or from another imaging method. Also, an expert opinion may provide: for the required 

ground tmth. Then, on the basis of the ground truth, the system may aiatomatically 
25 determine a set of parameters which allows for an improved reproduction i.e. for 

improved diagnosis result. The diagnostic image data and the corresponding ground 

truth and the thereto belonging parameter set is stored. 

Furthermore, as may be taken from above, the system may also be 

trained which may allow for an automatic selection of appropriate parameters. The 
30 system may continuously trained on the basis of training data which is constantly 

renewed or extended. Thus, by constantly training the system on the basis of a 
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constantly growing set of training data, the system may allow for improved diagnosis 
result. 

For example, the diagnosis may also be perfomoied on the basis of 
comparable data if, for example, there is no ground truth available. A comparable data 

5 set may be determined by, for example, determining a similarity between the current 
diagnostic image data and image data stored in the memory. Such image similarity may 
be determined on the basis of global image features. Also, such image similarity may be 
determined on the basis of statistical properties of image intensities. Also, the method or 
system may be adapted such that a plurality of comparable images is taken and the most 

10 appropriate parameter set and respective variations are determined on the basis of an 
interpolation between the parameter sets that were valid for all sufficiently similar input 
images. For the further processing of the diagnostic image data, the parameter set 
corresponding to the comparable image which has been foxmd may be used 
subsequently. 

15 Advantageously, in addition to the visualization of confidence intervals, 

according to the above exemplary embodiment of the present invention, a standardized 
feedback mechanism is provided which offers similar appearance and user interaction 
for different embodiments of the invention. Furthermore, advantageously, the 
confidence intervals are made explorable by user interaction. For example, uncertainty 

20 in quantification, registration and segmentation algorithms may be mediated by means 
of simple sliders. Moving the slider fi^om left to right may result in the presentation of 
algorithmic results that cover the full confidence range. 

Fig. 3 shows an exemplary embodiment of a display that may be 
displayed on display 4 according to the present invention. 

25 As may be taken firom Fig. 3, the display may, for example, be 

performed in Windows format having a region 30 where the diagnostic image data is 
displayed. Furthermore, there are provided three button 32, 34 and 36. Button 32 allows 
for an automatic trust feedback. According to a variant of this exemplary embodiment 
of the present invention, by activating the trust button 32, the image data currently 

30 displayed on area 30 and the corresponding data that may automatically be forwarded to 
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a service port from which it may be distributed for further use. This forwarding may, for 
example, be performed automatically by email. 

The distrust button 34 serves to express the distrust of the user in the 
segmentation result achieved by means of the current parameter set. Advantageously, by 
5 activating the distrust button 34, the diagnostic infiage data displayed on area 30 and the 
corresponding parameter set may be forwarded to the service port for further use. 

Reference numeral 38 indicates the segmentation result. Line 40 
indicates the diagnosis result determined on the basis of the current parameter setting. 
Dotted lines 42 visualize the confidence intervals. Line 40 represents the diagnostic 
10 result when the slider 44 is centered. The user may then explore the confidence interval 
of the applied algorithm. The segmentation boarder evolves according to the slider 
position in the confidence range indicated by dotted lines 42. The quantification results 
change as well from a 95% secured minimal value to a maximal value. Button 36 **train 

■ 

known standard" may allow the user to manually correct the contour. According to a 
1 5 manual correction of the contour, the system may then adjust parameters of the 

segmentation accordingly. 

The "train known standard" button 36 may be used to adjust technical 
parameters that mfluence the algorithms performance. Whenever a second opinion, for 
example an expert's subjective visual inq)ression or a ground truth measurement is 

20 available, these results can be trained into the algorithm. Using the "learning from 
examples" paradigm as, for example, described in Bredno J. et. al.: "Automatic 
Parameter Setting For Balloon Models" Procs. SPIE 3979, pp 1 185-1 195, 2000 which is 
hereby incorporated by reference, technical parameters are adjusted so that the ground 
truth is reproduced by the algorithm. According to a variant of this exemplary 

25 embodiment, this learning from examples may further include a regression analysis that 
is able to adjust technical parameters based on global input's data characteristics, such 
as described in Lehmann T.M. et. al.: "Texture-Adaptive Active Contour Models" 
LNCS 2013, pp 387-396, 2001 which is hereby incorporated by reference. 

Advantageously, according to the present invention, a multi-step method 

30 is provided which is applicable to a variety of image- and signal processing methods 
that are the base of clinical applications and which are apphed to computer-aided 
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diagnosis. Results of such algorithms include quantitative measurements, 
transformation parameters for registration, geometric segmentation results or others. 
According to the present invention, the trust of users towards such systems and the 
diagnosis results may be improved. 
5 It should be noted that the present application is not limited to the field 

of computer-aided diagnosis but may also be applied to similar fields like intervention 
planning, navigation support and outcome control where quantitative information is 
extracted firom medical data acquisitions. 



